Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SO-4122: index schema migration support #1249

Merged
merged 10 commits into from
Dec 5, 2023
Merged

SO-4122: index schema migration support #1249

merged 10 commits into from
Dec 5, 2023

Conversation

cmark
Copy link
Member

@cmark cmark commented Dec 5, 2023

This PR adds a very basic, index schema migrator functionality to aid existing indices to migrate to a newer schema version.

@cmark
feat(index): commit annotation based schema migrator configuration
a1bfa6e
@cmark
feat(index): commit progress on index schema migration logic
60c5e7d
@cmark
chore(index): remove unused SearchContextMissingException
48d1c2d
@cmark
feat(index): rename Migrator annotation to Schema
082e5dc
@cmark
feat(index): use dynamic index names in EsDocumentSearcher and Writer...
2245e5f 
...so that they can be used even if the actual index name is different
for a mapping than the real one (useful when running migrations).
@cmark
feat(index): add support to directly persist JSON objects in writers
ada1457
@cmark
feat(index): change type to ObjectNode as every document is an object
31d78f0 
Introduce init(Searcher) method to allow caching of services and global
data when migrating an index.
@cmark
chore(index): correct typo in comment
652e6b8
@cmark
feat(index): initial version of the mapping schema migrator
6713a7a
@cmark cmark added the feature label Dec 5, 2023
@cmark cmark requested review from apeteri and nagyo December 5, 2023 12:35
@cmark cmark self-assigned this Dec 5, 2023
@codecov Codecov
Copy link

codecov bot commented Dec 5, 2023 •
edited
Loading

Codecov Report

Attention: 61 lines in your changes are missing coverage. Please review.

Comparison is base (86564c1) 47.87% compared to head (6713a7a) 47.96%.
Report is 9 commits behind head on 9.x.

❗ Current head 6713a7a differs from pull request most recent head ddd20cd. Consider uploading reports for the commit ddd20cd to get more accurate results

Files Patch % Lines
...m/b2international/index/es/admin/EsIndexAdmin.java 80.12% 21 Missing and 10 partials ⚠️
...m/b2international/index/es/admin/IndexMapping.java 74.19% 5 Missing and 3 partials ⚠️
...com/b2international/index/es/EsDocumentWriter.java 73.33% 4 Missing ⚠️
...b2international/index/mapping/DocumentMapping.java 75.00% 4 Missing ⚠️
...m/b2international/index/es/EsDocumentSearcher.java 72.72% 3 Missing ⚠️
...ational/index/migrate/DocumentMappingMigrator.java 0.00% 3 Missing ⚠️
...erge/SnomedComponentRevisionConflictProcessor.java 0.00% 3 Missing ⚠️
...rnational/index/revision/DefaultRevisionIndex.java 0.00% 2 Missing ⚠️
...national/index/revision/DefaultRevisionWriter.java 0.00% 1 Missing ⚠️
...om/b2international/index/revision/StagingArea.java 75.00% 1 Missing ⚠️
... and 1 more
Additional details and impacted files 
@@             Coverage Diff              @@
##                9.x    #1249      +/-   ##
============================================
+ Coverage     47.87%   47.96%   +0.09%     
- Complexity    13797    13833      +36     
============================================
  Files          1940     1942       +2     
  Lines         94835    94937     +102     
  Branches      10948    10956       +8     
============================================
+ Hits          45402    45540     +138     
+ Misses        46416    46373      -43     
- Partials       3017     3024       +7

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

apeteri
apeteri requested changes Dec 5, 2023
View reviewed changes
commons/com.b2international.index/src/com/b2international/index/es/EsDocumentWriter.java Outdated Show resolved Hide resolved
commons/com.b2international.index/src/com/b2international/index/es/admin/EsIndexAdmin.java Outdated Show resolved Hide resolved
commons/com.b2international.index/src/com/b2international/index/es/admin/EsIndexAdmin.java Outdated Show resolved Hide resolved
commons/com.b2international.index/src/com/b2international/index/es/admin/EsIndexAdmin.java Outdated Show resolved Hide resolved
commons/com.b2international.index/src/com/b2international/index/es/admin/EsIndexAdmin.java Outdated Show resolved Hide resolved
commons/com.b2international.index/src/com/b2international/index/es/admin/EsIndexAdmin.java Outdated Show resolved Hide resolved
commons/com.b2international.index/src/com/b2international/index/es/admin/EsIndexAdmin.java Outdated Show resolved Hide resolved
nagyo
nagyo requested changes Dec 5, 2023
View reviewed changes
Copy link
Member

@nagyo nagyo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good to me, some minor suggestions in the comments!

commons/com.b2international.index/src/com/b2international/index/es/admin/EsIndexAdmin.java Outdated Show resolved Hide resolved
commons/com.b2international.index/src/com/b2international/index/es/admin/EsIndexAdmin.java Outdated Show resolved Hide resolved
@cmark cmark marked this pull request as ready for review December 5, 2023 16:48
apeteri
apeteri approved these changes Dec 5, 2023
View reviewed changes
Copy link
Member

@apeteri apeteri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

nagyo
nagyo approved these changes Dec 5, 2023
View reviewed changes
Copy link
Member

@nagyo nagyo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@cmark cmark merged commit e57082a into 9.x Dec 5, 2023
2 checks passed
@cmark cmark deleted the feature/SO-4122-index-migration-support branch December 5, 2023 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nagyo nagyo nagyo approved these changes

@apeteri apeteri apeteri approved these changes

Assignees

@cmark cmark

Labels
feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@cmark @apeteri @nagyo